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DISCUSSION 

Frank V. Thompson, Assistant Superintendent of Schools, Boston. — 
A year ago we were deliberating on the question whether the element of voca- 
tion in education seriously threatened our cherished cultural ideals. You may 
recall that it was hard work to raise an issue — and that apparently all the 
opinions expressed were upon the negative. In a short year, educational 
attention has focused upon the question of educational products — their extent, 
reality, and values. Contemporaneously in industry and business has come 
the movement known as scientific management. It is a favorable sign to note 
that education today is so sensitive and responsive to the spirit of progress 
outside academic walls. Modern education feels that all the influences, inven- 
tions, and improvements of promise affecting extraneous agencies are of vital 
importance to the school. Modern education is human, no longer merely 
scholastic. 

Education today wishes to be efficient, as industry is efficient; education 
wishes to know what the product is, and to gauge the time, quantity, and value 
elements. We speak in education today of vocation and culture. We are 
going to define the terms, and we are going to see that we actually produce 
what we say we are doing. We are going to determine the ratio of product 
and time; we are going to define standards; we are going to eliminate waste — 
in a word, we are going to try to be efficient, in conformity with the world-wide 
ideal of efficiency. But efficiency we know is a relative term. There is only 
one way to determine efficiency, and that is by measurement. Education 
cannot assume an efficiency-ideal without adopting its concomitant, namely, 
measurement. Hence our particular consideration this morning rests upon the 
topic of tests of school efficiency. 

Indeed, it may be admitted that the school has always sought to gauge 
its products; the accumulation of school statistics has grown to be enormous. 
We have our figures dealing with examinations, attendance, promotions, 
retardations — in a word, a very definite system of educational measurement. 

Why, then, the need of a new system of measurement, it may be asked ? 
The answer probably is this: The measurements employed in the past were not 
fundamental. These measurements dealt with standards peculiar to the 
school; they did not vitally touch life. Our standards have lacked objec- 
tivity and could not be interpreted in the common terms of life-activities and 
experiences. On the other hand, our school standards have not even achieved 
their own limited ends, and the defects and limitations of school statistics are 
becoming increasingly apparent, particularly when measured by newer stand- 
ards. School statistics today stand properly indicted. 

Now, like all movements receiving temporary emphasis in education, the 
present exaltation of measurement will need to run the gauntlet of its predeces- 
sors. There will be scoffers and zealots, and those who will wisely profit. It 
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is easy to point out evident dangers. Measurement in education presupposes 
commensurable quantities, and many of the best products of education are 
incommensurable. There are spiritual and there are material products of 
education. Over the first of these there will be ever the veil of mystery and 
doubt. Man spiritually has always been the enigma of existence, and will 
continue to be so. We can never plot the curve of genius nor measure the unit 
of inspiration. But a vast amount of educational effort is measurable, and a 
wiser method of measuring it is to be welcomed. 

Our discussion today deals with the measurable products of education. 
In devoting our attention to such matters, we do not for a moment lose our 
sense of values, of the higher, constant, and general aims of education. These 
values are measured only in the lives of citizens, in the creations of genius, and 
in the greatness of nations. It is best to begin on things which all agree are 
measurable. It may be that we shall see the extension of the possibility of 
measurement over many fields of what we now consider the spiritual side of 
education, but to assume that all activities are measurable would be inexpe- 
dient, to say the least, and, to my mind, unfortunate. 

But in the undisputed field of measurable quantities, I believe that we 
are bound to overhaul the present means of measurement of educational facts 
and products. The day of the educational engineer is at hand. Methods, 
programs, procedure, time-allotments, the whole machinery of education, will 
be subjected to investigations and tests which will upset many time-honored 
assumptions. The day of educational opinion will go — and the sway of domi- 
nant educational personalities will be much limited. Education today is an 
incipient science. If the educator of the future takes advantage of his oppor- 
tunities we may have, perhaps, an actual science. But I am not so confident 
as some of the speakers seem to be, of seeing the certainty of education being 
an exact science in my own day and generation. 

Now I think I have challenged, in a way, the apparent opinion of both 
the papers that all products of education are measurable. It is possible to sub- 
mit certain tests of spiritual values; but such tests must be suitable to the 
nature of the spiritual product. We naturally come to philosophical questions 
when we consider any fundamental question. The difficulty is this: when 
you attempt to measure the purely spiritual by something purely material 
you have no common unit. There are no common relationships. Many 
products of education are objective, and here measures may be applied with 
profit. 

Frank E. Spatjlding, Superintendent of Schools, Newton, Massachusetts. 
— As a school administrator charged with the responsibility of the general 
direction of an educational enterprise whose function it is to take a certain type 
of raw material and turn that raw material into a finished, or improved, 
product, I am greatly interested in the problems that have been raised and 
discussed here this morning, the problems of establishing standards for the 
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definite measurement of educational results. The school administrator, even 
more, perhaps, than the teacher, appreciates the importance of comparing 
results secured under different conditions, of determining their relative values. 
It is, indeed, a fundamental part of administrative work to make such com- 
parisons constantly. At present, the school administrator is forced to guess 
at values, to rely chiefly upon his individual opinion, just as teachers do when 
they mark a pupil's paper. So, as a school administrator, I want to express 
my hearty appreciation of the purpose and the spirit of the studies which these 
pioneers are undertaking, studies that are to furnish definite standards of 
educational measurement. 

I want to speak briefly of a standard to which educational products are, 
must be, constantly referred by the school administrator, and to suggest the 
great importance of definite educational measurements in connection with this 
standard. I refer to the financial standard. What does it cost to produce a 
given result ? Of two results produced by equal expenditures of money, which 
is the more valuable ? What investments of money bring the largest returns ? 
How may the money available for the maintenance of a public educational 
enterprise be used to produce the most valuable results ? These are extremely 
practical educational questions from the administrative standpoint. Whether 
or not we like even the suggestion of referring educational values to the expense 
in dollars and cents that went into the production of those values, such reference 
we must make constantly in practice, if we would conduct efficiently this 
productive enterprise in which we are engaged. 

Now if objective standards of measurement can be worked out and applied 
in place of the standard of personal opinion, we shall be able to tell with a degree 
of certainty how to expend educational funds to the best advantage; we shall 
be able to determine what expenditures bring the largest returns. Let me 
illustrate with a few concrete facts. 

As a part of the educational enterprise for which I am somewhat respon- 
sible, there are two high schools, an academic high school and a technical high 
school. In these two high schools are taught a large number of different 
subjects, grouped into eight or ten distinct courses. The cost of providing a 
definite unit of instruction varies quite markedly in the different subjects and 
in the same subjects in the two schools. Taking as the definite unit five 
recitations a week for one year — that is, two hundred recitations — for a single 
pupil, we find that the cost, at the present time, of this amount of instruction 
in history is in one high school $10. 20, in the other $11.80. Is the product 
m one school worth 15 per cent more than that in the other? It costs that 
much more. In one school it cost last year $ 10. 60 to give a pupil two hundred 
recitations in science; in the other school this year it is costing $16.69 to 
furnish the same amount of instruction in the same subject. Is the product 
secured at this higher expense worth 57 per cent more than the product secured 
at the lower expense ? It is costing that much more. In the same school it is 
costing $8.20 to provide one unit of instruction in commercial subjects — 
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stenography, typewriting, bookkeeping, etc. — while it is costing $17.60 to 
provide the same unit in household economics, and $17 . 93 in shopwork. Are 
the educational products worked out in the kitchen and the shop worth 115 per 
cent and 119 per cent more respectively than the product of the commercial 
rooms ? They are costing that much more. 

At the present rates it is costing $201 . 90 for the instruction necessary to 
put a pupil through the classical course, $206.24 to put one through the 
scientific course, and $219 to put one through the general course, in the Newton 
High School; it is costing $269.77 to put a boy and $281.63 to P u t a girl 
through the technical course, $336.92 to put a boy and $319.08 to put a girl 
through the extra-technical course, and $191.67 to put a pupil through the 
commercial course in the Technical High School. Here are extreme variations 
of over 75 per cent in the cost of instruction in the different courses in the same 
school. How do the educational values produced compare? When we are 
in possession of such objective standards of measurement as the discussion this 
morning leads us to anticipate, we shall be able to answer these questions and 
scores of similar questions with confidence. But while we are awaiting these 
standards, we must still answer these questions as best we can. The questions 
will not wait. We may refuse to face such questions squarely and to answer 
them consciously, relying upon such grossly imperfect measurements as we are 
now able to apply — and I suspect this is what most of us do; we may refuse 
even to formulate such questions; yet we cannot avoid answering them in 
some way in our practice of school administration. 

When the differences in cost are quite extreme, we do not find it so difficult 
to bring ourselves to a decision concerning the relative values of the products 
secured from the expenditures. When it was discovered a few years ago that 
it was costing about a hundred dollars to furnish instruction in Greek to a 
single pupil — two hundred recitations — we did not hesitate long in our answers 
to such questions as these. Which is more profitable, to teach ten pupils 
English, nine pupils history, eight pupils German, ten pupils mechanical 
drawing, six pupils household economics, or one pupil Greek ? In spite of some 
sentimental prejudice in this matter, to which I am willing to confess, I could 
find but one answer to all these questions, and we are now investing our money 
in educational products produced by other studies than Greek. Are we 
making a mistake ? Perhaps so. Hasten the day when we shall have objective 
standards of measurement on which we can rely! 

Edward M. Hartwell, Secretary, Statistics Department, Boston. — 
When asked to take part in this discussion I was in doubt whether I ought to 
accept the invitation, as I could only guess at what the papers in the program 
might suggest to a mere statistician. However, I decided to chance it, as I 
had other reasons for being interested in a meeting of this Association. 

I was a teacher before I became a worker in statistics. Moreover, as a 
teacher I was one of those who enjoyed the privilege of attending as a student, 
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in 1874, the second session of the Summer School in Zoology, which Professor 
Louis Agassiz had started the year before. And in 1875 I was with Professor 
Shaler's Harvard School of Geology at Cumberland Gap, in Kentucky. 

Thanks to the initiative of Professors Agassiz and Shaler, Harvard Uni- 
versity, a little later, established laboratories and museums for teachers of this 
vicinity, thus providing the material out of which this Association has grown 
and developed. 

Even in those days criticism of the secondary schools and their curriculum 
found voice, although no one had the hardihood to avow a belief in scientific 
pedagogy. I remember a meeting of the High-School Teachers' Association 
at which the professor of English in the Institute of Technology, then in its 
callow youth, vigorously attacked the study of Greek and such like rubbish, 
which attack was warmly repelled by President Eliot. 

Wendell Phillips argued that the public schools ought to make a place in 
their curriculum for vocational training; but his doctrine was so novel and 
heretical that he made but slight impression on public opinion. Meanwhile 
the times have greatly changed. Latterly dissatisfaction with the ideals and 
methods of the secondary schools, and of our colleges and universities, has 
grown apace. Criticism of results has become clamorous, and discordant as 
well. The practical man, so called, is especially insistent in his demands for 
results, such results as the wayfaring man, though only self-educated, can com- 
prehend and approve. 

Go to, cry the apostles of business efficiency and cost-accounting; let us 
standardize the schools in conformity with the maxims of the market-place 
and the manufactory, so that we can measure and appraise their output. 

Doubtless the scientific admeasurement of measurable quantities may be 
made to yield valuable criteria of the efficiency of certain methods and pro- 
cedures pertaining to school life and administration. But the measurements 
must be made by patient and skilful measurers, who can be depended on to 
winnow and classify their results and not jump at conclusions. 

My inclination is to welcome the attempt to extend and render more fruit- 
ful the field of school statistics, in which precisely determined, commensurable 
and comparable data are none too common. Such attempts compel my sym- 
pathy and respect. As a statistician one is most largely interested in things 
that can be counted and measured, and then be so classified and discriminated 
that their quantitative and causal relations may be determined and inter- 
preted. But I must confess that the glowing accounts of the rapid spread, in 
the last few years, of comprehensive statistical investigations of schools and 
scholars does not elate me overmuch. Supposedly professional statisticians 
not infrequently do such queer things that the prospect of a possibly too-rapid 
multiplication of amateur school statisticians is not inspiring. 

Only a few weeks since, one of the Boston newspapers printed a disturbing 
statement that, according to the United States Bureau of the Census, the death- 
rate of Boston for 191 1 was 18. 5 per thousand of population: the highest of 
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the eight largest cities in the country. The Census people early in the year, 
doubtless to impress the public with their up-to-dateness, published a series of 
death-rates for 1911. But they gave out the Boston death-rate without waiting 
to ascertain the actual number of deaths in 191 1. They estimated the number 
of deaths for the last third of the year on the basis of the deaths previously 
recorded. The actual death-rate for the year, using the same estimated 
population, figured out at 1 7 . 1 , or 1 . 4 less per thousand living than the Census' 
figure. Moreover, due allowance for the very exceptional month of July was 
not made in the Census estimate of deaths. Deaths in Boston last July 
numbered 1,270 — of which 216 were from the effects of heat — or 345 more than 
in July, 1910, when there were no fatal heat-strokes. Official estimates of 
population on a basis for computing annual death-rates are sometimes aston- 
ishingly wide of the mark. I knew a health officer who for three consecutive 
years figured his death-rates on an estimated population of 541,000 (which 
figure he obtained from a newspaper) ; but the United States Census enumer- 
ators could find but 508,957 people in the city in the third year, which was a 
Census year. In another large city for three consecutive years 405,000 was 
used as a basis for computing the death-rate. A year later the Census enumera- 
tion amounted to only 325,902, so that the death-rates for the preceding three 
years were from 3.6 to 3.8 per thousand too low. 

When such things are done in the dry tree, what may we not expect in the 
green? 

Undoubtedly there is room for improvement in the matter of keeping 
school records and the compiling of attendance statistics. There is good 
reason for the suspicion that the Massachusetts statistics of school attendance 
have been misleading, particularly as regards the numbers entering and leaving 
school within a year. The original Massachusetts School Register was devised 
in the thirties by Mr. Lemuel Shattuck, of Boston, a versatile and competent 
statistician; but he gave no place in it for recording the death of a pupil. 
Tardiness and absence bulk large in our school records; but they afford us no 
data for estimating, much less computing, the mortality of either city or coun- 
try children. So far as I know, they do not enable one to measure the loss 
of time in the schools caused by disease in general or by school-diseases in 
particular. If Mr. Ayres knows of any city in the country where the school 
authorities or the vital statisticians record the deaths of school children, 
I should like to know its name. I submit that it is high time that the ad- 
vocates of uniform school records should set about securing data to serve as 
a basis for comprehensive and trustworthy mortality and morbidity statistics 
of the school population. 

It is not a very difficult matter to pile up figures as to school accommoda- 
tions and the physical conditions of schools in certain cities. We have recently 
been informed, as the result of a hygienic survey, that 65 per cent of the school 
population is below par — does not come up to the normal standard. Dr. 
Richardson, a prolific English writer on hygiene, used to say that he never saw 
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a healthy child. Well, one can define and refine his standard of health so that 
ideally proportioned and endowed persons would be too few to furnish an 
adequate supply of museum specimens. All of us know a good many healthy 
children. Any child or man may be pronounced healthy so long as he can do 
his daily work easily as often as the day's work comes around. When one 
scrutinizes the statistics which stigmatize two-thirds of the Boston school 
children as below the normal standard in physique and health, much comfort 
may be had from the fact that the defects are not organic or constitutional, 
but of a relatively unimportant sort, like neglected teeth, squint, et cetera. 

Many a well-meant statistical investigation has broken down because the 
schedules of inquiry were ill-devised, too elaborate, or too minute. Granted 
that the data for tabulation are trustworthy, the classification and arrangement 
of data in congruent groups calls for insight and skill no less than for industry 
and patience. Sound statistical work is so expensive at every stage that thrifty 
shopkeepers and efficient inquirers are prone to condemn it, because, according 
to their lights and standards, results appear incommensurate with their cost. 

When I hear that state-wide investigations of schools or school children 
have been completed in a year, I cannot forbear wondering how many of the 
investigators were experts, where they came from, how much money they spent, 
and what they really accomplished. 

I believe in putting searching questions when there is a prospect that 
they will elicit honest and intelligent replies. There is urgent need for 
determining whether school life stunts or stimulates the normal growth and 
development of the average school child in respect to body and mind. The 
ever-increasing demands of the schools upon the tax-levy make it imperative 
that economic leaks in their administration shall be located and stopped. At 
present opinion is divided and uncertain as to the functions of the schools and 
what they should accomplish. In these fields of inquiry exploration must 
necessarily precede research. Thus far we have had more exploration than 
research. 

In view of the present state of the arts of exploration and research, I am 
rather glad that the supply of well-equipped school hygienists and statisticians 
hardly warrants school authorities in adopting ambitious schemes of investiga- 
tion on a large scale; else we should have a deluge of ill-digested and misleading 
comparative statistics. The public is inordinately fond of the vain comparative, 
particularly in its interpretation of statistics relating to population, health, 
and wealth. How often one finds statistics adulterated or distorted to enable 
a community or an institution to brag over its compeers! 

It is natural that popular education should be influenced by the currents 
and cross-currents of popular opinion. Just now popular opinion is running 
strongly in favor of industrial training and commercialized education. Those 
who are responsible for the guidance of popular education will doubtless be 
forced to engage in self-examination and to investigate in various directions 
in order to meet the criticisms and demands of the apostles of business effi- 
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ciency and to attempt to frame their answers in terms of units of energy and 
units of cost. 

Here in Massachusetts, without due investigation or a well-reasoned plan 
of reconstruction, the State Board of Education has been revolutionized. Thus 
far there appears to be no consensus of expert opinion as to the meaning of 
industrial education, where it should begin and end, or how it should be con- 
ducted. Our present predicament seems to me largely owing to the impatient 
spirit, muddled notions, and hustling methods of the business world. It is 
only recently that business men have awakened to the commonplace truths 
that underlie the gospel of business efficiency. So long as most investigations 
of political corruption lead up to a business man or a group of them, one may 
be pardoned for some hesitation in accepting business men, as such, at their 
own valuation. It may be seriously doubted, too, whether, at the present 
stage of their mental and spiritual development, they are likely to prove the 
most intelligent and helpful reformers of our school system. 

It seems to me that the essential problems of education are problems con- 
cerning growth and development ; and that in our attempts to evaluate and 
appraise the results of instruction and training we should strive to adopt such 
standards of measurement and comparison as are applicable to living beings 
and their activities, and eschew as inapplicable the criteria by which the 
efficiency of the builders and operators of dead machines is properly judged. 
Statistical problems that relate to the development and actions of human beings 
and of potential and social institutions are the problems which appeal to me 
most strongly, because I was a biologist before I became a statistician. My 
plea is that we should discriminate in our investigation and consideration of 
educational matters between methods and criteria that are applicable to living 
mechanisms and their activities and those which pertain to the realm of the 
inventor, the engineer, and the manufacturer. 

Lest I seem to be a reactionary and a skeptic, let me express my interest 
in the papers presented here this morning. The first embodied an attempt to 
devise a workable method of assigning quantitative values to impressions and 
qualities. While recognizing the interest attaching to the problem as an exer- 
cise in logic and philosophy, I confess that I should not dare to put the proposed 
scale to a practical use, without receiving careful instruction from Dr. Thorndike. 

With Mr. Ayres's main thesis, as I understand, I am in substantial accord. 
His paper is a very suggestive one. I hope I may have an opportunity to study 
it carefully. Possibly such study may modify my belief that a preliminary 
campaign of education as to the nature and limitations of the statistical method 
is desirable, if trustworthy and illuminative results are to be secured by such 
extensive investigations as Mr. Ayres advocated. 

Stratton D. Brooks, Superintendent of Schools, Boston. — I observe that 
the audience is getting nervous. Perhaps it is well to close this discussion, 
except for the privilege of the chairman to contribute his mite. 
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I feel that the school administrator is confronted with a very difficult 
situation, in that his cost is measurable and his product is not measurable. 
The public, the business man, the taxpayer knows exactly how much school 
supplies cost, and how many are furnished, but we are unable to prove to him 
that the school is delivering a product that is as good as it was forty years ago. 

I have been interested, of course, in calling the attention of some gentlemen 
to the records in my office of the schools of long ago. In fact, we were investi- 
gated, and the investigator came with the attitude that the school was not 
delivering the product that it was bound to deliver. I pointed out to him the 
forty-five volumes, more or less, that I have, in which I have every paper that 
was written in an examination long ago in the schools of Boston; and I 
challenged him to take any book at random, and I would guarantee that any 
pupil in the eighth grade could write better than any pupil in the file. He did 
not believe it until he looked at the book; but it was true; and he admitted 
it at once. He went through it and found page after page upon which the 
pupil had answered every question with "I don't know." He went farther, 
and found that whenever a pupil made a mistake in an examination, the same 
mistake occurred in every paper through the book, which indicated a very 
radical difference from the methods of education of today; namely, that in that 
day they learned the answer by heart, and if they knew it, all right; and if 
not, they wrote down "I don't know." If they had learned it by heart, every 
last one of them wrote it alike, and incorrectly. 

I want to emphasize that the investigations may possibly help us. It may 
enable us to show that after all the school has a measurable product in some of 
its phases. We may be able to call in some of the experts who possess ability 
to inspect the schools; we shall have sufficient material to be able to approach 
the business man with a certain demonstration that we know something about 
our business, whatever the statistics are, and as bad as we know they are. 
Even an investigation, as bad as some of them are, may have this advantage — 
that the conclusions, if any are made, will be so confusing that the business 
man cannot understand them ; and it will therefore be a valuable asset in our 
defense of the public schools. 

I remember that in 1851 a superintendent of schools for Boston was 
appointed, and it was enacted that he should be "a genial influence on our 
schools, but should not otherwise interfere with their operation." In 1012, we 
have investigating committees which seem to be irritating influences on the 
schools but do not otherwise interfere with their operation. What shall we do 
about it ? What is the use of taking time and energy for investigation and 
having nothing come of it ? What is the use of coming to this discussion and 
going home and not doing something about it ? The fact is, that, as Mr. 
Hartwell said, we ought not to be so anxious to know what the other fellow is 
doing, or whether he is better or worse than we are, but what we are doing 
ourselves and why we are doing it. If these things can be reduced to terms of 
measurement, we want the measure made by those men who have the time and 
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ability to figure it out and bring it to us. If Professor Thorndike can show us 
a plan whereby we can measure our product, we want it; we want instructions 
how to use it. For those things not yet reduced to a money standard, we want 
the judgment standard at least. Here is the difficulty. Some of you believe 
you cannot measure thought progression in English composition. Professor 
Thorndike says they are beginning to work out a measure. The trouble is, 
that those teachers who say it cannot be measured will nevertheless mark one 
composition 76 and another 76.5. They have done it for years; refusing 
graduation to pupils who got 74. 5 and graduating those who got 75. Now to 
be sure, we may not get the exact standard ultimately, but we are going to get 
something more intelligent at least than estimation from the judgment stand- 
ard. Eighty per cent of the people might agree that one composition was the 
best one in the lot, and that another was second best, and so on; but go per 
cent of the teachers want to mark on the basis of whether it expresses what 
the teacher has in his mind. The essential point in the estimation of the com- 
position is what the child had in his mind and whether what the child said was 
80 per cent or 90 per cent of what the child knew. Since none of you can know 
what that particular child knew about that particular thing, you can never 
know whether he expressed 70 per cent or 80 per cent of his own thought. The 
chances are that he expressed 100 per cent of what was in his mind while he 
did the writing. He may have known more about it the day before or the day 
after, but when he was writing, he was putting down all he knew then. 

One or two words more and I have done. Some time ago, we tried to find 
out with regard to the financial condition of the Boston public schools because 
we wanted more money than the authorities in the City Hall and in the Legis- 
lature would allow us. We found no city in the United States that had sta- 
tistics about finances that were anywhere near true or had anything to do with 
the real facts. They had averaged the evening-school pupils at $5 each, grade 
pupils at $35, high-school pupils at $100 each, and struck a general average. 
And they knew so little as to believe that that would tell the truth, when it was 
readily possible, by changing the percentage of pupils, to spend $2 per pupil 
more on every pupil in the school and have a smaller average at the end of the 
year; or, reversing it, to spend $2 less and have a larger average. Con- 
sequently, we have attempted to revamp that somewhat. Those of you who 
are principals of elementary schools in Boston know that we know and you 
know exactly the amount spent in such schools and upon what items. That 
has come to be a very valuable form of statistics. 



